NB Convert 6.0 support for export #14177
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 9 commits
September 29, 2020 14:49
|
Our functional tests just recently broke for exporting a notebook or the interactive window to python script. Did some poking around at it seem that version 6.0 of nb convert is a pretty recent release and made breaking changes (which is why the 6.0 major bump) to the templating system. In particular the old null.tpl that we used to use for our export base is not available anymore. We have to use a new null.j2 in a new location. |
IanMatthewHuff
requested review from
DonJayamanne,
rchiodo,
joyceerhl,
DavidKutu and
greazer
IanMatthewHuff
commented
Sep 30, 2020
| ) { | ||
| this.templatePromise = this.createTemplateFile(); | ||
| } | ||
| ) {} |
There was a problem hiding this comment.
I moved the construction of the template files to the point where they were needed instead of construction. Didn't see the point of slowing down activation for a feature that's probably not a common every day use scenario. Plus triggering an export is already a user action with an expected wait time, so seems fine to build the file then.
IanMatthewHuff
commented
Sep 30, 2020
| // Use the jupyter nbconvert functionality to turn the notebook into a python file | ||
| if (await this.jupyterExecution.isImportSupported()) { | ||
| if (nbConvertVersion) { |
There was a problem hiding this comment.
This could be a bit simpler if we didn't worry about users changing the jupyter interpreter while running. But I figured that might be a possibility. So I tested that this does support moving your Jupyter interpreter to one with nbconvert 5 to one that uses nbconvert 6 and back without restarting.
Codecov Report
@@ Coverage Diff @@
## main #14177 +/- ##
==========================================
- Coverage 59.94% 59.92% -0.02%
==========================================
Files 705 706 +1
Lines 39101 39158 +57
Branches 5667 5681 +14
==========================================
+ Hits 23440 23467 +27
- Misses 14429 14454 +25
- Partials 1232 1237 +5
Continue to review full report at Codecov.
|
added 2 commits
September 30, 2020 04:45
DavidKutu
approved these changes
Sep 30, 2020
DonJayamanne
requested changes
Sep 30, 2020
rchiodo
reviewed
Sep 30, 2020
rchiodo
reviewed
Sep 30, 2020
src/client/datascience/jupyter/interpreter/jupyterInterpreterSubCommandExecutionService.ts
Show resolved
Hide resolved
DonJayamanne
approved these changes
Sep 30, 2020
|
Kudos, SonarCloud Quality Gate passed!
|
rchiodo
approved these changes
Sep 30, 2020
IanMatthewHuff
added a commit
to IanMatthewHuff/vscode-python
that referenced
this pull request
Oct 1, 2020
rchiodo
added a commit
that referenced
this pull request
Oct 1, 2020
* Add an optional onUpdated event to the iterator returned by ILocator.iterEnvs(). (#13950) In order to ensure that ILocator.iterEnvs() can finish as fast as possible, we add a side-channel event for each iteration. This event fires any time an already-yielded env info object is updated (e.g. reduced/merged or resolved/completed). The update only relates to operations triggered by that particular iteration. ILocator.onChanged remains separate and only relates to when the locators finds a new env, notices one was removed, or that one was otherwise fundamentally changed. There are two similar approaches we could take for this update event. Either we added the event as a property of IPythonEnvsIterator or we change the return value of ILocator.iterEnvs() to be a 2-tuple ([PythonEnvsIterator, Event<PythonEnvUpdatedEvent>]). We took the property approach since most of the time callers won't need to worry about the update events. * fully bump raw kernel (#13941) Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> * Virtualenvwrapper locator (#13895) * virtualenvwrapper locator * skip os-specific tests * Test getDefaultVirtualenvwrapperDir * Update src/client/pythonEnvironments/discovery/locators/services/virtualenvwrapperLocator.ts Co-authored-by: Karthik Nadig <kanadig@microsoft.com> * add pathExists check to WORKON_HOME dir * Change how the locator works + move util func out * Fix functional tests * utils unit tests * Update locator tests * Stub getUserHomeDir instead of getDefaultetc Co-authored-by: Karthik Nadig <kanadig@microsoft.com> * Clean up eslint errors (#13951) * Clean up eslint errors * Fix linting and address comments. * Add a legacy DI adapter for the Python envs component. (#13858) This allows us to start using the new discovery code in the extension. * Extension api for DataScience (#13923) * Extension API * Api for DS * Fix installing ipykernel into interpreters for raw kernel (#13959) * Fix kernel and server name missing in certain situations (#13974) * Fix kernel name and server name * Fixup server name for remote situations * Add some functional tests * Add news entry * Extension API to return path to debugpy for DS (#13973) * Use the component adapter in the extension. (#13869) This allows us to start using the new discovery code in the extension. The key thing is to be careful not to regress in the available functionality. So for now we have disabled use of the component. When we feel comfortable with it we can enable it by setting the default for `ComponentAdapter.enabled` to `true`. This PR involves small fixes to a large number of files due to API changes. The adapter is actually used (injected) in the following files: src/client/interpreter/interpreterService.ts src/client/interpreter/helpers.ts src/client/pythonEnvironments/discovery/locators/index.ts src/client/pythonEnvironments/discovery/locators/services/condaService.ts src/client/pythonEnvironments/discovery/locators/services/windowsStoreInterpreter.ts * Disable split views of custom editors (#13985) * Fix backup storage by looking at the options correctly (#13983) * Fix backup storage by looking at the options correctly * Fix backup by being more explicit * Only linux tests are failing. Hopefully fix them * update test plan for ipykernel (python repo) (#13993) * Find interpreter based on hash in kernelspec of nb metadata (#13856) * Pyenv locator (#13996) * Pyenv locator * Skip tests per platform * Wrong pyenv path order in ternary * Add description * Autoformat venv locator * Revert "Autoformat venv locator" This reverts commit 5c8c4ab. * Add links * Windows-specific fixes * Typo * merge * Revert "merge" This reverts commit ee0cad9. * Remove cwd while running using isolate script (#14014) * Remove cwd while running using isolate script * Run inplace. * Fix formatting * Upgrade isort to 5.5.3 (#14035) * Modify environment info worker to support new type and to work with resolver (#13997) * Modify environment info worker to support new environment type * Do not replace existing fields and return a new object * Modify cache to carry deferred instead * Change worker to return interpreter information instead * Handle error and rename * Fix bug with interpreterInfo.py * Code reviews * Rename old PythonEnvInfo type to InterpreterInfoJson * Remove cross process lock (#14036) * Add API to get language server from external extensions (#14021) * Ideas for lang server API with python extension * Working with new API * Minimize API surface * Fix up tests for intellisense * Fix unit tests * Put back custom editor service * Remove unnecessary changes for service registry * Code review feedback * Move connection type to types file * Fix failing virtualenvwrapper Windows unit tests (#14012) * Fix tests * Typo Co-authored-by: Karthik Nadig <kanadig@microsoft.com> Co-authored-by: Karthik Nadig <kanadig@microsoft.com> * make sure that we run our show code when a custom editor is shown (#14022) * make sure that we run our show code when a custom editor is shown * Trigger notebook load in the constructor, not show Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> * Rely on AppInsights API for exceptions (#13878) * Fix path * Actually fix settings * Add news * Add test * Format * Suppress 'jediEnabled' removal * Drop survey first launch threshold * Telemetry for exceptions * Remove custom exception telemetry * Remove unused * Fix HTML escaping to match what Jupyter does (#14038) * Basic idea * add some functional tests * Add news entry * Fix functional tests * Have dependabot ignore pytest and py (#14039) Otherwise it doesn't understand the Python 2.7 restrictions for those dependencies. * Added environments reducer (#13953) * Add environments reducer * Added tests * Use path.join to construct paths * Code reviews * Correct dummy implementations and adjust tests * Modify resolveEnv() * Rename to a general parentLocator * Merge release back to master (#14059) * Port ipykernel install fix to release (#13975) * Fix installing ipykernel into interpreters for raw kernel (#13959) * update news Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> * Merge in changes to release (#13976) * Up release version for new release (#13928) * Up release version * Update changelog * Update changelog * Workaround test issue (#13930) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * More tests failing with gpu error (#13935) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * Two more version changes * Fix kernel and server name missing in certain situations (#13974) * Fix kernel name and server name * Fixup server name for remote situations * Add some functional tests * Add news entry * Delete news file * Port two fixes to the release branch (#13995) * Disable split views of custom editors (#13985) * Fix backup storage by looking at the options correctly (#13983) * Fix backup storage by looking at the options correctly * Fix backup by being more explicit * Only linux tests are failing. Hopefully fix them * Fixup changelog Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> * add jedi-language-server to 3rd party notices (#13977) * add jedi-language-server to 3rd party notices * move license from distribution to repository file * disable test_discover_complex_default and (#14024) test_discover_complex_doctest * Upgrade isort to 5.5.3 (#14035) (#14037) * prepare release (#14042) * fixed annoying warnings (#14049) * update version Co-authored-by: Ian Huff <ianhu@microsoft.com> Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> Co-authored-by: Rich Chiodo <rchiodo@users.noreply.github.com> Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> Co-authored-by: Kartik Raj <karraj@microsoft.com> * Fix object sort order in tools tests (#14050) * Fix object sort order in tools tests * rebase with main * Fix formatting * Update ubuntu images for GPU issue fix (#14064) * Update ubuntu images for GPU issue fix * Try with ubuntu-20.04 * Added environments resolver (#14019) * Modify environment info worker to support new environment type * Change worker to return interpreter information instead * Modify resolveEnv() * Code reviews * Code reviews * Move stuff * Add Windows unit tests to the PR pipeline (#14106) * Do not opt users out of the insiders program if they have a stable version installed (#14091) * Remove code that opts users out of the insiders program * News entry * Code reviews * Do not quote isolated in exec module (#14108) * Do not quote isolated in exec module * Revert "Do not quote isolated in exec module" This reverts commit b9fa04c. * Revert "IPyKernel install issue with windows paths (#13667)" This reverts commit 23725ab. * Update cell output and metadata using Edit API (#13737) * Fix unit test broken by recent revert (#14122) * revert VSC_PYTHON_CI_TEST_VSC_CHANNEL changes (#14109) * revert VSC_PYTHON_CI_TEST_VSC_CHANNEL changes * Revert "revert VSC_PYTHON_CI_TEST_VSC_CHANNEL changes" This reverts commit 6ed8f61. * Remove force 1.48 on Test VSC channel * Merge environment and compare environments (#14026) * Add some heuristic functions * Support merging. * Clean up identifier code * Tweaks and fixes * More clean up. * versions tweak * Address comments * more comments * Fix merge issues. * Fix more merge issues. * Add PythonEnvInfo-related helpers. (#14051) This PR adds some basic helpers that we use in a subsequent PR. The following small drive-by changes are also included: * drop PythonEnvInfo.id property * make some internal helpers public * Escaping fix broke a number of things (#14145) * Fixes for escaping * Push a comment ot start PR again * Cache task is failing * Remove cache task * Not fixing so just put back cache task * Auto assign datascience issues (#14147) * Change name of event used to activate extension (#14153) * Fixes to streamed output in native notebooks (#14158) For #13611 * Fixes to streamed output * added tests * Fixes to for escaping of output in native notebooks (#14159) * Look for `data science` labels when auto assigning issues (#14161) * Environment info cache class (#14065) * Add persistent storage external deps * PythonEnvInfoCache class + tests * Instantiate & initialize cache class in createAPI * Add extra test for flush() and initialize() * Env cache fixes: storage key + find() result check * Update src/client/pythonEnvironments/common/externalDependencies.ts Co-authored-by: Kartik Raj <karraj@microsoft.com> * Use areSameEnvironment in getEnv * Don't ping persistent storage for every initialize * No need to export CompleteEnvInfoFunction * PythonEnvInfoCache doc comment * Rename createGlobalPersistentStoreStub to get... * Preemptively drop id key (#14051) * Return deep copies * IPersistentStore wrapper around IPersistentState * Use correct areSameEnvironment + fix stub * Remove obsolete comment * getEnv -> filterEnvs * Remove stubbing of areSameEnvironment * Update areSameEnv * Move IPersistentStateFactory registration to registerForIOC * Revert "Move IPersistentStateFactory registration to registerForIOC" This reverts commit edc6ce5. * Don't instantiate nor initialize cache for now Co-authored-by: Kartik Raj <karraj@microsoft.com> * Update Component Governance notes in the release plan * Merge point release back into main (#14171) * Port ipykernel install fix to release (#13975) * Fix installing ipykernel into interpreters for raw kernel (#13959) * update news Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> * Merge in changes to release (#13976) * Up release version for new release (#13928) * Up release version * Update changelog * Update changelog * Workaround test issue (#13930) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * More tests failing with gpu error (#13935) * Try different version of VS code in release * Change to make it use the actual variable * Use a real version * Two more version changes * Fix kernel and server name missing in certain situations (#13974) * Fix kernel name and server name * Fixup server name for remote situations * Add some functional tests * Add news entry * Delete news file * Port two fixes to the release branch (#13995) * Disable split views of custom editors (#13985) * Fix backup storage by looking at the options correctly (#13983) * Fix backup storage by looking at the options correctly * Fix backup by being more explicit * Only linux tests are failing. Hopefully fix them * Fixup changelog Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> * add jedi-language-server to 3rd party notices (#13977) * add jedi-language-server to 3rd party notices * move license from distribution to repository file * disable test_discover_complex_default and (#14024) test_discover_complex_doctest * Upgrade isort to 5.5.3 (#14035) (#14037) * prepare release (#14042) * fixed annoying warnings (#14049) * Cherry pick to address path issues. (#14125) * Do not quote isolated in exec module (#14108) * Do not quote isolated in exec module * Revert "Do not quote isolated in exec module" This reverts commit b9fa04c. * Revert "IPyKernel install issue with windows paths (#13667)" This reverts commit 23725ab. * Fix unit test broken by recent revert (#14122) * Port escape fix to release branch (#14133) * Fix HTML escaping to match what Jupyter does (#14038) * Basic idea * add some functional tests * Add news entry * Fix functional tests * Update changelog * update version and changelog (#14139) * Escaping fix broke a number of things (#14145) (#14154) * Fixes for escaping * Push a comment ot start PR again * Cache task is failing * Remove cache task * Not fixing so just put back cache task * remove leftovers * import pytest * stop skipping python tests Co-authored-by: Ian Huff <ianhu@microsoft.com> Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> Co-authored-by: Rich Chiodo <rchiodo@users.noreply.github.com> Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> Co-authored-by: Kartik Raj <karraj@microsoft.com> Co-authored-by: Karthik Nadig <kanadig@microsoft.com> * NB Convert 6.0 support for export (#14177) * Windows store locator (#14162) * Initial commit for windows store locator * More tests * Simplify locator * Tweaks * Test fixes * Fix tests * A different way of fixing escaping (#14186) * Move escaping to just output * Add some tests to verify round tripping * Fixup test for rountrip and make roundtripping actually work * Add news entry * Add to manual test file * Fix streaming problem and add more to the test * Fix traceback unit test * Fix problem caught by functional tests :) * Another functional test catch * Fixes to escaping of output in native notebooks (#14199) * Display description in kernel picker (#14196) * Bump node-fetch to 2.6.1 (#14181) * Bump pytest from 6.0.1 to 6.1.0 in /news (#14190) Bumps [pytest](https://github.com/pytest-dev/pytest) from 6.0.1 to 6.1.0. - [Release notes](https://github.com/pytest-dev/pytest/releases) - [Changelog](https://github.com/pytest-dev/pytest/blob/master/CHANGELOG.rst) - [Commits](pytest-dev/pytest@6.0.1...6.1.0) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Messed up merge commits Co-authored-by: Eric Snow <eric.snow@microsoft.com> Co-authored-by: Ian Huff <ianhu@microsoft.com> Co-authored-by: Ian Huff <ianhuff@CEIDCCEVIPSVC01.redmond.corp.microsoft.com> Co-authored-by: Kim-Adeline Miguel <51720070+kimadeline@users.noreply.github.com> Co-authored-by: Karthik Nadig <kanadig@microsoft.com> Co-authored-by: Don Jayamanne <don.jayamanne@yahoo.com> Co-authored-by: Graham Wheeler <gram@geekraver.com> Co-authored-by: Kartik Raj <karraj@microsoft.com> Co-authored-by: Joyce Er <joyceerhuiling@gmail.com> Co-authored-by: Mikhail Arkhipov <mikhaila@microsoft.com> Co-authored-by: Brett Cannon <brett@python.org> Co-authored-by: David Kutugata <dakutuga@microsoft.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
DonJayamanne
pushed a commit
that referenced
this pull request
Oct 30, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
For #14169
package-lock.jsonhas been regenerated by runningnpm install(if dependencies have changed).